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INTRODUCTION 


T he French philosopher Paul Valery observed with nostalgia that “the trouble with our times 
is that the future is not what it used to be.” This is particularly true in the realm of tertiary 
education, which is in great flux. A recent report published in the United Kingdom proposed 
the image of “an avalanche” to describe the radical changes affecting tertiary education in many parts of 
the world (Barber, Donnelly and Rizvi, 2013). Indeed, powerful transformative forces of three kinds- 
rupture factors, crisis factors and stimulus factors-are challenging tertiary education systems all over the 
planet. 

First, a growing number of rupture factors are at play in transforming the ecosystem in which tertiary 
education institutions are operating, drastically influencing how they perform their teaching and research 
functions. Among these rupture factors are technological innovations such as flipped classrooms for 
interactive learning, massive open online courses (MOOCS) reaching hundred of thousands of students 
all over the world, new forms of competition from for-proflt and corporate universities that provide 
professional qualifications closely linked to labor market needs, and new accountability modalities like 
the global rankings, which allow us to measure and compare the performance of universities across all 
continents (Salmi, 2013). 

Second, as a result of the 2007 financial downturn, the tertiary education sector in most regions of the 
world has been affected by serious crisis factors. In the United States, for example, the level of public 
support for tertiary education has been reduced substantially in nearly every state-48 out of 50 over 
the 2008-13 period-under the combined impact of the economic recession, federal mandates to fund 
other sectors such as healthcare, and the reluctance to increase state taxes (Miller, 2013). In Europe, 

1 3 out of the 20 university systems that the European Universities Association has been monitoring 
since the beginning of the financial crisis experienced overall budget decreases in real terms between 
2008 and 2012, nine of them of more than 10 percent (EUA, 2013). The cuts have been even more 
severe throughout the developing world, with the aggravation of falling household incomes and soaring 
graduate unemployment rates. 

The third type of factors worth mentioning are the stimulus factors that refer, in contrast to the 
previously mentioned crisis dimensions, to the availability of significant additional funding for tertiary 
education in a small number of countries where governments consider that the role of universities in 
support of innovation and growth is so important that they deserve extra resources. In most cases, the 
package of additional funding has come in the form of an “Excellence Initiative” designed to strengthen 
the leading universities of the countries concerned, China, Denmark, France, Germany, Russia, South 
Korea, and Taiwan to mention a few recent examples. 

Against this background of complex forces at play, tertiary education enrollment has continued to 
increase rapidly in most parts of the world. The proliferation of tertiary education institutions has 
generated concerns about the quality and relevance of the programs offered, especially in the case of 
private providers, and put additional pressure on quality assurance agencies. Ironically, even in situations 
of significantly reduced public funding for tertiary education, governments have not lightened their 
demands on quality assurance agencies and tertiary education institutions. If anything, it seems that the 
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role of government has become more intrusive in recent years and the accountability requirements have 
grown significantly. 

In this context, the main objective of this report is to analyze how the interplay among governments, 
quality assurance agencies and tertiary education institutions has evolved in the past decade. After a brief 
summary of the history of quality assurance, the second part of the report documents the evolution of the 
role of the State vis-a-vis quality assurance agencies in recent years, identifying significant developments, 
key issues and new challenges. While the report takes a global perspective, it does not in any way attempt 
to provide an exhaustive review of trends in the relationship between governments and quality assurance 
agencies. The third part of the report explores the growing importance of new forms of accountability, 
such as rankings and measures of learning outcomes. The report concludes by proposing a few principles 
to help reach an appropriate balance between autonomy and accountability as they apply to the 
relationship between governments and quality assurance agencies. 


THE QUIET QUALITY ASSURANCE REVOLUTION 
IN THE EARLY CENTURY 

U ntil the 1980s, tertiary education institutions in the United States, the United Kingdom and 
Commonwealth countries were the only ones in the world with a strong tradition of external 
quality assurance (QA). In the absence of a federal Ministry of Higher Education, the quality 
assurance function was carried out essentially through private accreditation agencies. This continues to be 
the prevailing modality today, unlike what happens elsewhere in the world. 

This all started to change in the 1980s and the 1990s, as most Organisation for Economic Co-Operation 
and Development (OECD) countries moved to establish some form of government-sanctioned quality 
assurance. Van Damme (2002) explains the rapid expansion of quality assurance in this period as the 
combination of at least five factors at play in many countries. First, the transition from elite to mass 
higher education led to concerns about a possible decline in academic standards. Second, employers lost 
confidence in the ability of tertiary education institutions to maintain the relevance of their programs 
in an increasingly competitive and global economy. Third, as governments reduced funding for tertiary 
education because of fiscal restrictions, they called for greater accountability in the use of public 
resources. Fourth, the growing competitiveness within and across tertiary education systems made it more 
important to have tangible measures of quality. Finally, stakeholders began to demand more transparency 
in tertiary education as far as quality was concerned. For example, the first college ranking published 
in 1983 by U.S. News and World Report re.s'pondtd to this societal preoccupation for more information 
about the performance of tertiary education institutions. 

Europe witnessed a considerable drive as a direct result of the Bologna process officially launched in 
1999. One of the most important dimensions of the Bologna process activities has indeed been the 
development and/or strengthening of quality assurance in all participating countries, based on the 
principles issued through the Berlin Communique in 2003, and the Standards and Guidelines prepared 
by the European Association for Quality Assurance in Higher Education (ENQA) and adopted by the 
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Ministers of Education/Higher Education in 2005 at the Bergen meeting. By 2008, most countries had 
a functioning evaluation or accreditation agency. The European Quality Assurance Register (EQAR) has 
been a strong factor in influencing QA agencies wishing to integrate the community of national systems 
recognized as being in compliance with the Standards and Guidelines. The successful convergence of QA 
regulations has been one of Bologna’s most noticeable outcomes. 

The former socialist countries of Eastern Europe and the Soviet Union have also been keen to participate 
in this process. Today, most of them have a quality assurance system in place, even though the capacity is 
still unequal, as reflected by the fact that many of the agencies from these countries have not been accepted 
as full members of ENQA. For example, the agencies from Albania, Armenia, Bosnia-Herzegovina, 
Georgia, Kazakhstan, the Former Yugoslav Republic of Macedonia, Romania, Russia, and the Slovak 
Republic are only affiliate members. 

Whereas only a minority of developing countries had a formal quality assurance system by the turn of the 
century, the QA movement has gained tremendous momentum in the past 1 5 years. In Latin America, 
the first quality assurance body was established in Mexico in 1991, followed two years later by a national 
accreditation agency in Colombia. In the following two decades, most countries in the region set up a 
national quality assurance body, with the exception of the Central American nations, which started with 
a regional accreditation agency. Today Uruguay is the only country in the region without any formal 
quality assurance and accreditation body, although the Ministry of Education is responsible for licensing 
new private universities. In the majority of cases the quality assurance agencies have been operating as 
independent bodies, but in three countries, Ecuador, El Salvador and Nicaragua, the government has 
maintained direct control of the quality assurance body. 

Asia and the Middle East have experienced a similar evolution. In South-East Asia, Indonesia took the 
lead in establishing a national quality assurance agency in 1994, followed over the next two decades by 
almost all the countries in the region. Today, Myanmar is the only tertiary education system without a 
formal external quality assurance department or agency. In the Arab world, the first decade of the new 
century saw the creation of quality assurance systems in most countries, 1 1 out of the 17 main countries 
in the region by 2009. Yemen was the twelfth nation, and today only five countries are without a formally 
established QA system. Two of them, Lebanon and Tunisia, are at an advanced stage in the setting up 
process. 

Africa is perhaps the region where the quality assurance movement has been slowest. By 2006, only six 
countries had a fully established quality assurance agency, Ghana, Nigeria and South Africa being the 
pioneers in that domain. In the past eight years, however, progress has been impressive, and today, 23 
countries have a national QA agency. The concluding declaration of a recent pan-African conference on 
quality assurance urges all countries that do not have with a proper QA system to put one in place as a 
matter of priority, especially in view of the growing importance of private tertiary education and e-learning 
(Jongsma, 2014). 

As a result of this worldwide phenomenon, countries can be characterized today as belonging to one of the 
following four categories: 
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• Advanced systems whose tertiary education institutions have well-developed internal quality 
assurance processes with a strong focus on quality enhancement, in line with national standards 
defined by the external quality assurance and/or accreditation agencies, often linked to the national 
qualifications framework; leading OECD economies would be in this category. 

• Well-established systems still relying predominantly on external quality assurance, where a 
significant proportion of tertiary education institutions do not fully meet the national quality 
assurance standards; many industrial and developing countries would be in this category. 

• Countries that are in the process of setting up and consolidating their quality assurance system; 
many developing countries and countries in transition in Eastern Europe and Central Asia would 
be in that category. 

• Countries that have not established a formal quality assurance system; these would encompass a 
few countries in Asia, the Middle East, Latin America and the Caribbean and the Middle East, and 
about two dozen countries in Africa. 

It is worth mentioning in this brief account of the modern history of quality assurance that, in parallel 
with the spread of national quality assurance agencies, the quality assurance movement has also taken on 
an important international dimension under the impulsion of the donor community (Wells, 2014). First, 
the German academic exchange agency, Deutsche Akademische Austausch Dienst (DAAD), undertook in 
the late 1990s to help the Spanish-speaking Central American countries build up their capacity to carry 
out accreditation, resulting in the establishment of a regional accreditation body {Consejo Centroamericano 
de Acreditacion de la Educacion Superior) in 2004. The premise of this project was that a regional agency 
could serve their quality assurance needs in a more effective manner than if each country would set up its 
own accreditation organism, from the viewpoint of achieving a critical mass of peer reviewers and reducing 
the risk of conflict of interest that small countries invariably face. DAAD is currently following a similar 
approach to support the development of quality assurance in West Africa. 

Second, alarmed by the impact of the rapid growth of cross-border education on quality, the United 
Nations Educational, Scientific and Cultural Organization (UNESCO) and the OECD teamed up 
between 2003 and 2005 to elaborate guidelines in support of all stakeholders involved in the provision of 
cross-border education programs (governments, providers, faculty members, students, quality assurance 
agencies, professional associations). The joint document resulting from this effort offers a synthesis of good 
practices and proposes tools to monitor and improve the quality and relevance of cross-border education 
in order to protect students from below-standard practices (UNESCO and OECD, 2005). 

In 2008, UNESCO also joined forces with the Council for Higher Education Accreditation (CHEA), the 
U.S. professional association in charge of recognizing accreditation bodies, to address the growing issue 
of diploma mills and accreditation mills, which offer and sanction worthless degrees affecting thousands of 
students not aware that they are exposed to fraudulent practices. Their 2009 Declaration sought to provide 
guidance to countries and quality assurance agencies keen on combating dishonest practices in tertiary 
education (CHEA and UNESCO, 2009). The official Communique of the 2009 World Conference on 
Higher Education further alerted Member States to the alarming phenomenon of degree mills (UNESCO, 
2009). 
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Third, the World Bank tertiary education strategy published in 2002 {Constructing Knowledge Societies) 
identified quality assurance as a global public good. 

Globalization and the growth of borderless education raise important issues that ajfect tertiary 
education in all countries but that are often beyond the control of any one government. Among 
the challenges of particular concern to countries seeking to build up their advanced human capital 
capacity are new forms of brain drain that result in a loss of local capacity in fields critical to 
development; the absence of a proper international accreditation and qualifications framework; the 
dearth of accepted legislation regarding foreign tertiary education providers. . . . 

The rapid development of virtual providers of tertiary education programs on a global scale, the 
increasing mobility of professionals across national borders, and the absence of quality assurance 
infrastructure and capacity in many developing countries make it important to establish an 
international framework that sets out minimum common standards worldwide (World Bank, 

2002, pp. 98 and 101). 

Following up on this recognition, the World Bank set aside grant resources to support the strengthening 
of INQAAHE {International Network for Quality Assurance Agencies in Higher Education) , the international 
umbrella association for quality assurance, and the development of quality assurance regional networks. A 
series of Development Grants helped establish the Asian and Pacific network (APQN) first, followed by the 
Latin American network (RIACES) and then by networks in the Arab (ANQAHE), African (AfriQAN) 
and Caribbean (CANQATE) regions. These grants culminated with a collaborative project, the Global 
Initiative for Quality Assurance Capacity (GIQAC), financed by the World Bank and managed by 
UNESCO, to consolidate the regional networks. More recently, the German cooperation program and the 
European Union have supported the development of AQAN, the ASEAN quality assurance network. 

Finally, it is important to note that this significant transformation of the tertiary education landscape 
has, interestingly, happened without too much controversy. In contrast to reforms in the areas of 
governance and financing, which have more often than not generated heated debates and encountered 
strong resistance in the academic community, the development of quality assurance has been widely 
accepted in most countries across all regions of the world. This can be explained by a combination of two 
major forces. First, the evolving balance in the steering of tertiary education systems has generally meant 
greater institutional autonomy and growing reliance on market mechanisms in exchange for increased 
accountability, including through formal quality assurance mechanisms (World Bank, 2002). Second, 
as described in the previous paragraphs, external factors at the global and regional levels have heavily 
influenced national and institutional behaviors. While the Bologna process has been a powerful vector 
of convergence in Europe, in developing countries the emergence of regional QA networks has acted as a 
strong catalyst. In addition, the expansion of cross-border education has reinforced the perceived need for 
regulatory and quality assurance measures in many corners of the planet. 
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RECENT TRENDS IN QUALITY ASSURANCE: 
THE CHANGING ROLE OE THE STATE 


Against the historical background presented above, this section documents recent trends in quality 
assurance requirements from the viewpoint of the evolving relationship between governments, quality 
assurance agencies and tertiary education institutions. As signaled earlier, the purpose is not to provide an 
exhaustive account of developments in all countries of the world in the past decade, but to give a sense of 
major trends through a few illustrative cases. 

Increased Scrutiny 

T he first important development could be described as tightening of quality assurance requirements, 
in the form of stricter regulations or even the closing of universities in countries where the 
quality assurance standards and process have proven insufficient to weed out below-standard 
institutions. This has often come as a result of the rapid proliferation of tertiary education institutions, 
growing suspicion towards the quality of private sector providers-especially the for-profits-and evidence of 
fraudulent practices. 

Latin America, the region that started earliest to put quality assurance in place, offers several examples 
of such shortcomings. As most countries opted initially to focus on program accreditation rather than 
institutional accreditation, they have found it difficult to keep up with the fast growth of programs, 
especially in the private sector. In Colombia, for instance, after more than 20 years of accreditation effort, 
accredited universities offer less than one percent of all graduate programs. The proportion of accredited 
institutions was seven percent in 2013, and the share of accredited programs was 13 percent. Interestingly, 
more private universities (13) had obtained the high level accreditation than public ones (only nine) 
(OECD, 2012). In Costa Rica there were only 63 accredited programs by 201 1, and only 74 in Bolivia. 

In 2012, the Government of Ecuador shut down 14 private universities out of the 26 that the 
Accreditation Agency had put on its list of worst performers. The same year, the Chilean government 
closed down a private university, Universidad del Mar, because of poor quality. In 2014, the Colombian 
Ministry of Education closed down several medical programs offered by a well-known private university, 
San Martin Foundation, because of serious concerns about the quality of teaching and lack of compliance 
with legal and tax requirements. The Ministry is contemplating similar actions against an undisclosed 
number of universities. Table 1 shows similar examples from other parts of the world, namely Eastern 
Europe, Africa and East Asia, illustrating concerns about inadequate quality and/or fraudulent practices. 
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Table 1 - Recent Examples of University Closures 


Countries 

Action 

Reasons 




Albania 

Closure of 18 universities in 
2014 

Issuance of fake diplomas 

Romania 

Closure of six universities in 
2013 and four in 2014 

Poor quality 

Ethiopia 

Closure of five universities and 
1 1 put on probation in 201 1 

Poor quality 

Kenya 

One hundred illegal institu- 
tions closed down in 201 1. All 
non-accredited universities 
might be closed down in 2014. 

Lack of authorization to oper- 
ate and poor quality 

Nigeria 

Closure of nine institutions 

Lack of authorization to oper- 
ate and poor quality 

Philippines 

Closure of an international 
business school in 201 1 

Poor quality 


Sources: 

Albania: httpiHwww. ansamed. infolnuova europaleninewsi countries! alhanial201 4l08l07lalbania-closes-l 8-uni- 
versities-for-issuinp-fake-diplomas de759b5e-lc73-4b57-9f9e-4b6805f4e802.html 

Romania: http:llwww.balkaninswht.coml enlarticlelromania-closes-four-private-universities 

Ethiopia: http:llethiopiaobservatory. coml201 1I09I06I1768I 

Kenya: http://www.universityworldnews.com/article.php?story=201 101282304044l4&query=shut+down 

Nigeria: http://www.aitonline.tv/post-nuc shuts down nine universities over illegal licence 

Philippines: http://www.rmanetwork.com/news/story/270293/pinoyabroad/news/pinoy-hk-students-cautioned- 
on-closed-economics-school 
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The Chilean case is perhaps the most striking. Recent scandals have completely undermined the credibility 
of the accreditation process. Considered for many years as one of the most advanced quality assurance 
systems in the region, the Chilean Accreditation Commission has lately been under fire with accusations of 
conflict of interest, lack of independence and insufficient professional rigor. One rector and a former acting 
chair of the Accreditation Commission were jailed under accusations of fraud and conflict of interest. The 
entire quality assurance system is undergoing a significant overhaul in the context of a new Accreditation 
Law presented to Congress. 

In Russia, acknowledging that the quality assurance efforts of the Ministry of Education have not been 
sufficient, the Government has begun to involve the Attorney General’s office to monitor the operation of 
public universities. More than 700 Russian universities are expected to undergo audits in the next year or 
so. 


In Hungary, the State has restricted the scope of responsibility of the Quality Assurance Agency in recent 
years. Established in 1993, the Hungarian QA body lost its full member status with ENQA in 2013. As a 
result of the new Higher Education Act passed in 201 1, it now plays only a consultative role with regard to 
accrediting new institutions or programs. It also lacks financial independence.' 

In the Netherlands, a government report published in 20 1 1 deemed existing quality assurance methods 
insufficient. This report also indicated that the Minister of Education should have authority to issue orders 
to a university’s board if serious failings were found. As a consequence, the Government decided to increase 
the role of the Ministry of Education’s Inspectorate, thereby undermining in part the authority of the 
official Accreditation Organization (NVAO) (Dutch Ministry of Education, Culture and Science, 2011). 

In Poland, the Government introduced amendments to the tertiary education legislation in 20 1 1 to shift 
the focus of the Polish Accreditation Committee (PKA) from inputs to learning outcomes, and from 
quantitative aspects previously regulated by national legislation to qualitative aspects.^ 

With regard to similar efforts to tighten quality assurance in East Asia, the South Korean experience 
is worth mentioning. In 201 1, the Ministry of Education, Science and Technology clamped down on 
low-quality private providers by “naming and shaming” 43 tertiary education institutions deemed of 
substandard quality, after a thorough evaluation of the country’s 346 private institutions (Kim, 201 1). As 
a result they lost access to the 143 million dollars of annual state subsidies that they had been receiving, 
and their students would not be eligible for student loans anymore. A few months later, in early 2012, the 
Ministry prohibited 36 universities and two-year colleges from receiving foreign students, again because of 
low quality. 

Along similar lines, the Government of Yemen decided in 2014 to stop financing scholarships for studies in 
private institutions in Malaysia, complaining about “poor educational quality and high costs” (Tan, 2014). 
More than 4,000 Yemeni students are currently studying in private Malaysian universities. 


1 http://unuw.mab.hu/web/doc/mabmin/External Review Report 300713.pdf. particularly pages 8-10, 34-35, 41. 

2 http://www.enaa.eu/wp-content/uploads/20l4/02/PKA-review Final-report-of-the-review-paneLpdf , page 9. 
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In Turkey, the new higher education law proposed by the Government in 2014 would give YOK, the 
national regulatory agency for higher education, additional powers regarding quality assurance for doctoral 
degrees. 


The United States is the last country example worth mentioning in this regard. In late 2013, the Senate 
questioned the effectiveness of the accreditors, wondering whether the various accreditation agencies were 
strict enough (Field, 2013). They called for the application of more rigorous standards and warned of 
potential conflicts of interest between the peer reviewers and the institutions being assessed. 

Consolidation and Transformation of 
QA Structures and Procedures 


A second, related trend is the transformation of QA structures and procedures to improve how the 

quality assurance functions are carried out. This has taken at least five forms: moving from program 
accreditation to institutional accreditation, making accreditation compulsory, decentralizing 
accreditation, merging existing QA agencies/departments into a single structure, and consolidating 
independent QA agencies. 


Some countries where the QA agency has been unable to keep up with the rapid growth of programs, 
especially in the private sector, have started to offer the option of institutional accreditation as a 
complementary way of ensuring quality. Colombia, for example, has moved in that direction, although 
the number of universities that have received institutional accreditation is still very small. In Mexico, the 
federation of private universities (FIMPES) relies on institutional accreditation to ensure the quality of 
education offered by its members. 


In the Middle East, the Saudi Ministry of Higher Education decided in 2013 to make accreditation 
compulsory rather than voluntary, as it had been in the past. Chile is considering the adoption of a similar 
measure as part of the general overhaul of the QA system mentioned earlier. 

Following the U.S. example of decentralized accreditation, Mexico has moved from a direct accreditation 
system to a distributed approach, whereby the government approves accreditation agencies responsible, in 
turn, for the external quality assurance functions of higher education institutions. Since 2002, the Council 
for Higher Education Accreditation (COPAES) has recognized accreditation agencies and associations 
according to the general criteria that it defined at the beginning of its mission {Lineamientos y Marco general 
para los procesos de acreditacion de programas de educacion superior). This explains why Mexico is perhaps 
the Latin American country that has advanced most in the area of quality assurance in its region, although 
progress has been mixed among the various types of institutions.^ 


3 The largest public universities (including UNAM) are autonomous and, as such, are not required to be accredited. The 
proportion of students enrolled in accredited programs ranges from 90 percent among the public, non-autonomous universities to 
50-60 percent among the technological universities and institutes, to only 23 percent among private universities. 

(Source: R. Tuirdn, “La Educacion Suprerior en Mexico, 2006-2012: un balance inicial”, 2013.) 
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Another dimension of transformation worth mentioning are efforts to rationalize and consolidate the 
oversight structure in some countries that had several QA agencies operating. In 2012, Ireland decided 
to integrate four agencies into the new Quality and Qualifications Ireland (QQI), replacing the Further 
Education and Training Awards Council, the Higher Education and Training Awards Council and the 
National Qualifications Authority of Ireland and incorporating the functions of the Irish Universities 
Quality Board.'* Similarly, in Austria, the new QA agency that was constituted in 2014 amalgamates the 
formerly separate quality assurance bodies responsible for universities and Fachhochschulen (polytechnics). 
In Malaysia, where the accreditation department within the Ministry of Higher Education National 
Accreditation Board (Lembaga Akreditasi Negara, LAN) was initially set up with jurisdiction over the 
private institutions exclusively, a second QA department (Quality Assurance Division, Ministry of Higher 
Education (QAD) was later created to look after the public institutions. In 2007, the Ministry brought 
both departments together under a single QA department called the Malaysian Qualifications Agency 
(MQA).5 

A variation of the merger option is found in countries that have traditionally kept the quality assurance 
functions under the direct responsibility of the main funding agency. In Pakistan, for instance, the Higher 
Education Commission, which plays the role of a higher education federal ministry, is also in charge of 
all quality assurance functions. Similarly, in Nepal, the University Grants Council is also responsible for 
quality assurance. 

Finally, some countries have made efforts to strengthen the independence of their QA agencies. In Croatia, 
for example, the government realized in 2009 that the Agency for Science and Higher Education (ASHE), 
the national agency responsible for quality assurance that had been set up in 2005, did not operate with 
sufficient autonomy because of the constraining legal framework. The new Act on Quality Assurance in 
Research and Higher Education redefined its responsibilities, making ASHE the only institution in Croatia 
competent to perform external quality assurance and establishing its Accreditation Council as a fully 
independent body composed of academics, students and other relevant stakeholders in the Croatian higher 
education sector.*^ 

Some of the Nordic countries, notably Denmark and Sweden, have experienced a pendulum movement in 
that respect. Both countries started with a relatively hands-off system focusing on quality enhancement at 
the institutional level, but then moved to program accreditation with more State control. Denmark, which 
has today two QA agencies, one for institutional accreditation and one for crosscutting systemic aspects, 
has gone back to an approach built on mutual trust, thereby guaranteeing more autonomy to higher 
education institutions. In Sweden, the Government forced a new QA system on the higher education 
sector in 201 1 without consultation with the institutions themselves. As a result, the new Swedish QA 
agency (HSV) got into trouble in the European area in 2012 for its perceived lack of independence from 
government. In addition, ENQA singled out HSV for not abiding by European standards and guidelines 
with respect to the impartiality of its assessment of internal quality assurance mechanisms (Myklebust, 

4 http'.Hwwiv. qqi. ielPmeslAhout- Us. aspx 

5 http://www. mqa.pov. my! 

6 http://ecahe.eu/vj/index.php/ASHE - Apency for Science and Hipher Education 
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2012). The government is now planning to allow HSV to operate more independently and to rely again on 
institutional accreditation. 

In France, a new High Council for the Evaluation of Research and Higher Education (HCERES) will soon 
replace the Agency for the Evaluation of Research and Higher Education (AERES), which was created 
in 2006. This change aims to increase the autonomy of French universities in alignment with the 2007 
governance reform, as HCERES would only perform evaluations and accreditations on request from the 
universities (Pain, 2013). 

Use of Funding Mechanisms for QA Purposes 

I t is also worth noting that some governments have encroached indirectly on the responsibilities of QA 
agencies through changes in the funding mechanisms used to allocate public resources among tertiary 
education institutions. In the Netherlands, for example, the funding formula rewards universities 
whose graduates finish their studies on time. Similarly, in the Canadian province of Quebec, successive 
governments have pressured the universities to become more efficient through quantitative indicators of 
completion rates (Salmi and Hauptman, 2006). 

In the United States, the number of states relying on some kind of performance-based funding approach 
has increased from seven to 33 since 2010 (College Productivity, 2012). The overarching objective of 
these initiatives is to reward those tertiary education institutions that make deliberate attempts to raise 
the proportion of students completing high quality programs. 

Similarly, performance contracts in Austria, Chile, France and Spain allow universities to receive 
additional funding in return for a commitment to fulfill a number of national objectives, including 
quality improvement, as measured by specific targets agreed between the relevant ministry of education 
and the institution. 

Deregulation 

I n contrast with those countries whose government has tightened up quality assurance structures, 
regulations and procedures, there are some cases where the national authorities have opted instead 
for relaxing the supervisory role of the quality assurance agency (Australia) or even relieving the 
public quality assurance agency of its exclusive responsibility to carry out evaluation and/or accreditation 
functions (England). In Australia, the definition of the role and responsibilities ofTEQSA, the quality 
assurance agency, has been in flux. Established in 2008 upon recommendation of the Bradley review of 
higher education that called for a strong national regulator with the power to register and close down 
established universities, TEQSA took over the duties of the previous agency (AUQA) with an extended 
mandate, combining the functions of the state-level quality assurance bodies and the national QA agency. 
But in 2013, the government decided to backtrack, stripping TEQSA of its full responsibilities and 
halving its budget for regulation and quality assurance (Hare, 2014). With this change, the government’s 
purpose was to lessen the regulatory burden on higher education institutions, emphasizing “risk, necessity 
and proportionality” as working principles.^ 


7 https://www.education.mv.au/teqsa-advisory-coundl-terms-reference 
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In England, a surprise announcement by the Higher Education Funding Council (HEFCE) in 
October 2014 revealed the government’s intention to open the door to private companies, charities and 
government agencies to take over the current responsibilities of the Quality Assurance Agency (Grove, 
2014). The main reason behind the proposed shift is the perception that, faced with a rapidly changing 
higher education world, the Quality Assurance Agency has been unable to cope with the need for 
frequent reviews, that it has been too lenient, and that the review process has become more formal than 
effectual. How the new system would be organized and would operate remains to be defined. 

It should also be noted in this context that the 2006 Services Directive of the European Union, which 
guarantees the freedom of public and private tertiary education institutions from any European country 
to establish a branch campus or franchise institution in any other country of the Union, infringes on the 
independence of national QA agencies. According to the Services Directive, the primary responsibility for 
quality assurance does not rest with the QA agency of the receiving country, but with the country from 
which the cross-border institution originates. 

Taiwan has recently experienced a milder version of deregulation. The Ministry of Education decided 
in 2012 to grant the most reliable universities the status of “self-accrediting” universities. This move 
was meant to respond to the wish for more institutional autonomy and the intention of the Ministry to 
help strengthen internal quality assurance systems. Self-accrediting universities are expected to have the 
capacity of assessing their own strengths and areas for improvement and to develop their own review 
standards. In addition, they would be given the authority to organize an external evaluation without 
prior review by HEEACT, the Taiwanese Quality Assurance Agency. 

Quality Assurance and Innovation 

O ne of the challenges faced by quality assurance and accreditation systems has been to find the 
right balance between enforcing standards in a reliable and consistent manner and allowing 
sufficient space for innovations in curriculum design and pedagogical approaches. Three recent 
developments, in India, Peru and Saudi Arabia, illustrate the tension between quality assurance and 
innovation that strict government regulations can create. 

In India, a number of institutions have introduced four-year undergraduate degrees in the past few 
years, moving away from the traditional British-like three-year programs (Narayan and Sharman, 2014). 
Among those who made the change are the University of Delhi, several Indian Institutes of Technology 
(IITs), the Indian Institute of Science and a few private universities. The rationale behind the change 
was to align their degrees closer to U.S. degrees, open the option of offering dual degrees with U.S. 
universities, and give their students the opportunity to acquire research skills during their undergraduate 
studies. 

However, India’s main regulatory body, the University Grants Council (UGC), has not recognized this 
change as a desirable move. Instead, in 2014 it instructed all the institutions that implemented the four- 
year degree to reverse course and stick to the standard three-year degree. The institutions concerned have 
tried to fight back through the courts or by asking the Minister of Human Resources to intervene in their 
favor, arguing that the four-year degrees were in line with international trends. 
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These moves could push the brightest students of India away from choosing a career in science. It 
could threaten innovation in higher education that is in bad need of an overhaul Experiments to 
improve education must be encouraged, especially if the premier institutes of the country are taking 
the lead. We can only know what works best if we attempt a variety of approaches. (Vishwesha 
Guttal, assistant professor at IISc) 

The IITs, which get their budget directly from the Central Government, not from the University Grants 
Council, have also argued that the UGC did not have jurisdiction over their academic affairs. But it looks 
doubtful that the UGC will reverse its stand. 

For several decades, the Peruvian tertiary education system was one of the least regulated ones in Latin 
America. The National Council that was responsible for authorizing new universities, CONAFU, 
consisted of university rectors and former rectors for whom it could be difficult to remain fully objective 
and independent when it came to licensing and accrediting new private sector competitors. To reduce the 
risk of conflicts of interest inherent to this setup, the government introduced a new Higher Education 
Law in 2014 that establishes a more independent and professional regulatory agency {Superintendencia 
National de Educacion Superior Universitaria - SINEDV) in charge of authorizing new institutions to 
operate for a fixed number of years. 

At the same time, however, the Law includes a number of restrictive clauses that seem to be out of sync 
with recent developments in the field of virtual and distance education. For example, the members of the 
Board of the Superintendencia must hold a PhD, but it cannot be an online degree. Similarly, no one who 
obtained her/his PhD online is eligible to become rector of a public or private university, the dean of a 
faculty, or even a full professor. The Law goes on to prohibit undergraduate degrees that are not taught at 
least 50 percent in a presential mode as well as continuing education programs that are shorter than five 
years, the equivalent of a regular full-time undergraduate degree. 

Several private universities, which have successfully pioneered the development of innovative virtual and 
continuing education programs in Peru, have claimed that political and commercial interests influenced 
the drafting of the Law. Whether this is true or not, the fact remains that these clauses against online 
education do not appear to be grounded in sound technical knowledge and run contrary to the current 
evolution of tertiary education delivery modalities, as reflected by the development of the MOOCs and 
the rapid expansion of online programs all over the world. 

A last example worth mentioning is that of Saudi Arabia. At the same time as the Government made 
accreditation compulsory rather than voluntary, as mentioned earlier, it decided that it would not 
recognize the international accreditation received by some programs, such as ABET accreditation for 
engineering programs. By forcing these programs to undergo the national accreditation process as well, 
the Ministry is sending a negative message to the more innovative universities that took the initiative to 
submit themselves to the scrutiny of international accreditors. 

More generally, universities in many countries complain that the evaluation/accreditation procedures 
have become so bureaucratic and cumbersome that they risk becoming a ritualistic process that adds 
little value to the actual quality of programs and pedagogical practices because of the focus on formal 
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compliance rather than on quality enhancement aspects. Anecdotal evidence from countries as diverse 
as Argentina, Australia, the Netherlands and the United Kingdom points to a lengthy and costly review 
process that does not always translate into meaningful guidance on areas that could be improved. 

A recent report prepared by the Australian Evaluation Agency, TEQSA, acknowledges the urgency 
of “reducing the excessively bureaucratic procedures, which strangles universities,” considering the 
numerous reports required every year by Parliament and Government at both the State and Federal levels 
(Maslen, 2013). As early as 2008, Peter Williams had observed from his vantage as president of ENQA 
that too many QA agencies had become fixated on processes rather than focusing on their original 
quality promotion purpose. He expressed concern about what he called “the ossification” of quality 
assurance (Williams, 2008). 
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Table 2 summarizes what can be considered as the most salient changes in the relationship between the 
State and the quality assurance agencies and/or tertiary education institutions. 


Table 2 — Typology of Government Interventions in Recent Years 


Types of Government Intervention 

Country Examples 

Creation of a QA department within Ministry or 

Brunei (2011), Lao PDR (2008), Nepal (2007), 

regulatory government body 

Vietnam (2004) 

Creation of independent QA agency 

Ecuador (2011), Greece (2005) 

Merger of various QA agencies 

Australia (2008), Austria (2014), 
Ireland (2011) 

More rigorous QA standards/requirements 

Chile (under review), Netherlands (2011), 
Turkey (2014) 

Making accreditation compulsory instead of 
voluntary 

Saudi Arabia (2013) 

Closing of low quality universities programs 

Colombia (2014), Ecuador (2012), 
Romania (2012) 

Naming and shaming/Cutting state subsidies 

South Korea (2011) 

Excessive bureaucratization 

Argentina, Australia, England, Netherlands 

Use of funding mechanisms to influence quality 

Austria, Chile, France, Quebec, many U.S. States 

Barrier to innovative practices 

India (2014), Peru (2014) 

Deregulation 

Australia (2013), England (2014) 


Source: Elaborated by the author 
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NEW FORMS OF ACCOUNTABILITY IN 
TERTIARY ED UCATION 


No good book was ever written on command, nor can good teaching occur under duress. And yet, 
conceding this, the fact remains that left entirely to their own devices academic communities are no less 
prone than other professional organizations to slip unconsciously into complacent habits, inward-looking 
standards of quality, self-serving canons of behavior. To counter these tendencies, there will always be a 
need to engage the outside world in a lively, continuing debate over the university’s social responsibilities. 

Derek Bok (1990) 

New instruments of accountability have appeared or been discussed in recent years, which could 
potentially affect the work of national QA agencies or complement the pool of information available to 
measure the performance and operation of tertiary education institutions. Of particular relevance in this 
context are the following modalities: 

• Student engagement surveys 

• Assessment of student learning outcomes 

• Labor market observatories 

• Rankings 

• Benchmarking 

Student Engagement Surveys 

F ollowing the example of the United States, where the first large-scale survey of student engagement 
(NSSE) took place in 2000, a number of countries have developed and implemented their own 
version of a survey to ascertain how students feel about the quality of teaching and learning in their 
institutions. Today, student engagement surveys are carried out regularly in Australia, Canada, Germany, 
Ireland, the Netherlands and the United Kingdom. Pilot surveys have also been undertaken in recent 
years in countries as diverse as China and South Africa. 

Continuing a movement that started in the 1960s with student evaluations of their teachers, student 
engagement surveys include not only subjective indicators such as the level of satisfaction of students, 
but also attempt to measure more objective aspects related to the degree of active engagement of students 
in interactive and collaborative learning activities (Ramsden and Callender, 2014). In countries where 
surveys of student engagement are conducted regularly, high school graduates tend to be better equipped 
to choose which college or university they would like to attend. Institutions that participate voluntarily 
in student engagement surveys can use the results for quality improvement purposes. 

Student engagement surveys face two challenges (Klemencic and Chirikov, 2014). First, some observers 
have questioned their validity and reliability with respect to the ability of students to make informed 
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judgments when asked to report learning gains and also in relation to the selection of the key factors that 
are supposed to determine student learning, assuming standards of institutional practice and student 
behavior (Porter et al, 2011). Second, not all stakeholders are ready for the kind of transparency that 
these surveys imply. For instance, many U.S. universities, including top-tier universities, continue to 
refuse to release their NSSE results (Salmi, 2007). 

Assessment of Student Learning Outcomes 


U nlike lower levels of education - primary and high school - the world of tertiary education 

does not have a long tradition of measuring learning outcomes. However, promising initiatives 
have emerged in recent years. In the United States, a growing number of institutions have been 
using one of three assessment instruments to try to measure added value at the undergraduate level: the 
ACT Collegiate Assessment of Academic Proficiency (CAAP), the ETS Proficiency Profile (EPP) and the 
Collegiate Learning Assessment (CLA). Similar instruments have been in use in other industrial countries 
such as Australia (Graduate Skills Assessment). 


A few Latin American countries— Brazil and Colombia for example— have also been pioneers in attempting 
to measure the acquisition of knowledge and competencies of undergraduate students. In Brazil, for 
example, when the late Paulo Renato, then Federal Minister of Education, introduced the Provdo in 1996 
as a voluntary test designed to compare the performance of similar programs across all universities, it was 
the first such national assessment system in the world. The Provdo consisted of a final course examination 
for undergraduate students that did not count towards the graduation of the students themselves but 
served to evaluate the results of their program and institution. The Provao was replaced in 2004 by a 
new test (ENADE), applied every three years to a sample of students, which examines the test scores 
of both first-year and last-year undergraduate students as an attempt to measure the added value of 
undergraduate programs (Salmi and Saroyan, 2007). Similarly, the Colombian Assessment Institute 
{Instituto Colombiano para la Evaluacion de la Educacion) has implemented two tests (SABER- 1 1 and 
SABER-PRO) since 2009 that measure students’ abilities at the start and end of their undergraduate 
education. 


In some cases, policy-makers have considered the opportunity of using students learning outcomes for 
quality assurance purposes. But these proposals have been met with caution by the tertiary education 
community, as illustrated by the controversy sparked by the 2006 report of the Spellings Commission 
on the Future of Higher Education in the United States. The report recommended measuring learning 
outcomes to complement the existing accreditation system. 

". . . by law, student learning is a core part of accreditation. Unfortunately, students are often the 
least informed, and the last to be considered. Accreditation remains one of the least publicized, 
least transparent parts of higher education— even compared to the Byzantine and bewildering 
financial aid system” (NACIQI 2007). 

Initiatives to measure student learning outcomes in an international perspective have also been received 
with little enthusiasm. In 2012, the OECD conducted a pilot experience to measure the achievement of 
generic competencies and the acquisition of professional skills in the areas of economics and engineering 
in the context of the AHELO project (Assessment of Higher Education Learning Outcomes). Even 
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though seventeen countries participated in the feasibility study and the pilot, the future of the project, 
presented as an alternative to the global rankings, remains uncertain (OECD, 2013). 

The recent emergence of private companies specializing in testing the work readiness of young graduates 
has introduced a new twist with respect to the assessment of student learning outcomes. In India, for 
example, several large multinational firms make it compulsory for anyone interested in applying for a job 
to take one of these professional tests. 

More than 1.5 million people in India have taken a test called the AMCAT (Aspiring Minds’ 

Computer Adaptive Test). The assessment measures aptitude in English, quantitative ability and 
logic. ...It also includes a variety of situational and judgment tests, which scrutinize personality 
types and soft skills to see how they might apply in specific fields (Fain, 2014). 

Labor Market Observatories 


A nother noteworthy development has been the establishment of Labor Market Observatories 
(LMOs) in a growing number of developing and transition countries, following the example 
of the many OECD countries that have employment observatories either at the supra-national 
level (European Union employment observatory), the national level (e.g.. Bureau of Labor Statistics in 
the United States, university-based AlmaLaurea observatory in Italy) and the sub-national level (e.g.. 
Learning and Skills observatory in Wales, OREF in France, Education-Employment Information system 
in Florida). The examples of Bulgaria, Chile and Colombia are worth mentioning in this context. 


Since 2012, the Bulgarian government has published detailed data on the labor market results of 
university graduates. Using data from the Registry of Tertiary Students and statistics from the National 
Social Security Administration, the Ministry of Education is able to provide a wealth of information on 
the types of jobs and levels of remuneration of graduates who left university in the previous five years. 
The database indicates, for instance, if the graduate found a job, if the position corresponds to the field 
and level of study, what type of employer she/he is working for, if the graduate has a permanent or 
temporary job, and the level of salary based on social security contributions. 


Supported by the Chilean Ministry of Education and jointly run by the School of Government of the 
private University Adolfo Abanez and the University of Chile’s Department of Industrial Engineering, 
Futuro Laboral aims to equip youths and students with academic orientation tools. Futuro Laboral 
provides information on the occupational situation of graduates of hundreds of professional and 
technical careers that represent 75 percent of technical and professional graduates. The information 
available to the public includes detailed data on salaries and employment opportunities. The portal 
displays, for each program of every tertiary education institution, detailed information on dropout rates, 
average time to degree, average earnings of the graduates after four years of graduation, current tuition 
fees for the program and accreditation status of the program. Employment and earnings data are not self- 
reported, but gathered from the database of the national tax revenue authority. Earnings are matched to 
the databases of graduates provided by the tertiary education institutions. The privacy of the information 
is maintained, as the tax service issues only the average values for each program in each institution, 
provided there are at least 25 individuals in each program/institution’s cohort for whom earnings data are 
available. 
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Graduados Colombia (Observatorio Laboral para la Educacion) was launched in 2005 and is managed by 
the Ministry of Education. It collects and presents information on the demand and supply of graduates. 
Students, families, tertiary education institutions, researchers and the productive sector have access 
to statistics on the academic level of the graduates of technical institutes and universities, the salaries 
they receive and the average time for finding the first job, as well as the cities where they work. The 
website serves as a tool for students trying to choose a career, and it is also useful for tertiary education 
institutions intent on renewing and adapting the programs they offer according to labor market needs. 
Graduados Colombia’s site provides links to job offers in Colombia and in other countries, as well as 
advice and tips on how to write and present a good resume. Visitors are able to look for the results of the 
graduate and employer surveys, as well as studies on specific disciplines and economic sectors. 

These three initiatives show relevant examples of labor market observatories that aim to provide a 
better understanding of and match among individuals’ professional aspirations, tertiary education, and 
occupational trends. As such, they help to address one of the main challenges of tertiary education: its 
relevance to individuals and societies. 

Rankings 

T he power of public opinion is nowhere more visible than in the growing influence of rankings. 
Initially limited to the United States, university rankings and league tables have multiplied 
in recent years, existing today in more than 35 industrial and developing countries (Salmi and 
Saroyan, 2007). 

The U.S. News [& World Report] rankings have become the nations de facto higher education 
accountability system— evaluating colleges and universities on a common scale and creating strong 
incentives for institutions to do things that raise their ratings. (Kevin Carey, 2006) 

While fully acknowledging their methodological limitations, it is undeniable that the rankings have 
often played a useful educational role by making relevant information available to the public, especially 
in countries lacking a formal system of quality assurance. In Poland, for example, when the transition to 
the market economy started in the early 1990s, there was a thirst for information about the quality of the 
rapidly proliferating private education institutions. This demand for information pushed the owner of 
Perspektyvy magazine to initiate the country’s first university ranking. Similarly, for many years the annual 
ranking published in Japan by the Asahi Shimbun fulfilled an essential quality assurance function in the 
absence of any evaluation or accreditation agency. In France, after the publication of the 2008 edition 
of the Shanghai ranking, the Secretary General of the national teachers union (SNESUP) complained 
that it was unfair to compare the performance of universities to a race at the Olympic Games, However, 
the French Minister of Higher Education declared a few days after the publication of the 2008 rankings, 
“[Tjhese lists of winners may not be ideal, but they do exist. . . . They show the urgency of reform for the 
[French] university” (Floc’h 2008). 

Some of the rankings include information from student engagement surveys and/or labor market 
observatories as key indicators. In Chile, for example, the country’s main weekly magazine {Que Pasa?) 
uses the results of Futuro Laboral to rank universities and programs every year on the basis of the labor 
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market outcomes of their graduates. Similarly, in Bulgaria, the Ministry of Education has developed a 
ranking that incorporates the labor market results of university graduates.* 

The proliferation of rankings has provoked intense reactions, ranging from disagreements about the 
very principle of rankings to criticism about the methodology used to produce them, boycotts, political 
pressure, and even court actions to stop their publication. 

The expansion of league tables and ranking exercises has not gone unnoticed by the various 
stakeholders and the reaction they elicit is rarely benign. Such rankings are often dismissed by their 
many critics as irrelevant exercises fraught with data and methodological flaws, they are boycotted 
by some universities angry at the results, and they are used by political opponents as a convenient 
way to criticize governments. (Salmi and Saroyan, 2007, p. 80) 

Despite several attempts to boycott the U.S. News and World Report and the MacLeans rankings in the 
United States and Canada, respectively, they remain very popular among students and parents trying to 
figure out how to choose among universities, colleges and study programs. 

Benchmarking 

B enchmarking, which is the process of comparing the performance of specific university programs 
or entire tertiary education institutions to similar programs or institutions, has been proposed 
as a more meaningful alternative to rankings (Salmi, 2013). Rather than assigning a rank order, 
benchmarking enables users to compare several programs/institutions against a series of performance 
indicators without relying on rank order numbers to designate the “best” among peer institutions. 

Unlike rankings, which tend to lead to a “race to the top,” benchmarking can provide a more tempered 
assessment of performance. 

The German Centre for Higher Education (CHE) offers one of the most comprehensive examples 
of benchmarking, even though people often refer, mistakenly, to the information available on the 
CHE website as the German ranking. CHE makes a large number of indicators of inputs, process and 
outcomes available-including the results of student and employer satisfaction surveys-distributed into 
three broad bands of universities: the top 25 percent, the middle 50 percent and the bottom 25 percent. 
Users can select which universities and indicators they would like to combine to conduct a more refined 
search. 

Elements of benchmarking have also reached the U.S. tertiary education scene in recent years. In 
September 2007, the American Association of State Colleges and Universities (AASCU) and the National 
Association of State Universities and Land-Grant Colleges (NASULGC) announced that they would 
start publishing key performance indicators through a Voluntary System of Accountability Program. 

The program was a reaction to the recommendations-and perceived threat-of the Spellings Commission 
report mentioned earlier. According to the plan released by the two associations, each participating 
university would use a common template-called a College Portrait-to post key data on costs, transfer 

8 http:/lrsvu. man. bg/rsvu.yHocale=en 
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and graduation rates, and student satisfaction. The program would also include an assessment of student 
learning from one of the existing tests. Among the sponsors of this proposal were the same university 
presidents who had decided to boycott the U.S. News and World Report rankings (Fischer 2007). 

In his 2013 state of the union speech, President Obama called for accountability changes that would 
include developing benchmarks for affordability and student outcomes as criteria for receiving federal 
student financial aid. Early in 2014, the College Affordability and Innovation Act of 2014 was 
introduced in the Senate. The bill foresaw the creation of an independent commission of students, 
academics and education stakeholders to develop minimum accountability standards for making college 
more affordable and more accessible for middle-income and low-income students and adding more 
learning value. Even though the proposed accountability website has encountered a lot of opposition 
from many quarters, it seems that the Obama administration is moving ahead with the project. 


CONCLUSION 

Higher education in 2014 may be getting what it deserves, paying the price of having been a 
law unto itself for too long. It is time to move beyond a defense of privileges and self-interest to 
constructive engagement with the public’s questions before the opportunity passes (Shirley Mullen, 
President of Houghton College, 2014). 

The organizing principle for accountability must be pride, not fear (NCAHE, 2005). 

This review of recent trends concerning the development of quality assurance from an international 
perspective has revealed several important aspects. While the establishment of national QA structures has 
become a universal movement, with fewer countries left without a proper quality assurance system every 
year, it is difficult to discern any single general trend in terms of the evolving relationship between the 
State and QA agencies. The country examples analyzed throughout the paper show a mixed picture, with 
some countries tightening the supervisory role of the State at the risk of compromising the independence 
of their national QA agency, others moving to grant more autonomy to their QA agency and/or the 
tertiary education institutions, and others carrying out significant structural changes with the aim of 
improving the effectiveness of their QA system. 

At the same time, new accountability mechanisms have emerged in recent years, complementing the 
traditional evaluation/accreditation role of QA agencies. Student engagement surveys, learning outcomes 
assessments, labor market observatories, rankings and performance indicators used in benchmarking 
exercises can all provide useful additional information for quality assurance purposes. The multiplicity 
of accountability mechanisms provides students, employers, government and society at large with more 
abundant and transparent data about the operation and results of tertiary education institutions. This 
also gives the opportunity to QA agencies to embrace a more comprehensive approach in fulfilling their 
quality enhancement mission. 
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In light of the analysis undertaken in this report, three principles of good accountability can be proposed. 
First, the relationship between the State and QA agencies must reflect a healthy balance between 
accountability and independence, with clear rules of engagement defined and agreed upon regarding 
both elements. While the State-and society at large-have a legitimate interest in ensuring the quality 
of tertiary education, especially in countries where private providers and/or cross-border providers have 
multiplied, QA agencies must enjoy sufficient autonomy to carry out their responsibilities in an effective 
manner. Excesses should be avoided on both sides. Governments must not allow politics and lack of trust 
to color their relationship with QA agencies, and the latter should not be too lenient towards below- 
standards providers or too rigid towards innovative institutions. 

Second, in order to make a meaningful difference, quality assurance should not focus mainly on the 
way tertiary education institutions operate, but on the educational results that they actually achieve. To 
use the distinction proposed by Stein (2005), procedural accountability, which is primarily concerned 
with rules and procedures, is less meaningful than substantive accountability, which focuses on the 
essence of the research, teaching, and learning in tertiary education institutions. It may be easier to 
monitor the first type of accountability, but it is without doubt more relevant to concentrate on the 
second, notwithstanding its complexity and the difficulties involved in measuring the acquisition of 
competencies, student learning outcomes, and added value. 

Finally, the most effective accountability mechanisms are those that are mutually agreed upon between 
QA agencies and tertiary education institutions. Agreement ensures a greater sense of responsibility for 
the evaluation and feedback process and fuller ownership of the quality assurance instruments. 
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